Metrics and Benchmarking for Parallel
نویسندگان
چکیده
The evaluation of parallel job schedulers hinges on two things: the use of appropriate metrics, and the use of appropriate workloads on which the scheduler can operate. We argue that the focus should be on on-line open systems, and propose that a standard workload should be used as a benchmark for schedulers. This benchmark will specify distributions of parallelism and runtime, as found by analyzing accounting traces, and also internal structures that create diierent speedup and synchronization characteristics. As for metrics, we present some problems with slowdown and bounded slowdown that have been proposed recently.
منابع مشابه
Metrics and Benchmarking for Parallel Job Scheduling
The evaluation of parallel job schedulers hinges on two things: the use of appropriate metrics, and the use of appropriate workloads on which the scheduler can operate. We argue that the focus should be on on-line open systems, and propose that a standard workload should be used as a benchmark for schedulers. This benchmark will specify distributions of parallelism and runtime, as found by anal...
متن کاملBenchmarking Explicit State Parallel Model Checkers
This paper presents a set of benchmarks and metrics for performance reporting in explicit state parallel model checking algorithms. The benchmarks are selected for controllability, and the metrics are chosen to measure speedup and communication overhead. The benchmarks and metrics are used to compare two parallel model checking algorithms: partition and random walk. Implementations of the parti...
متن کاملSMaSH: a benchmarking toolkit for human genome variant calling
MOTIVATION Computational methods are essential to extract actionable information from raw sequencing data, and to thus fulfill the promise of next-generation sequencing technology. Unfortunately, computational tools developed to call variants from human sequencing data disagree on many of their predictions, and current methods to evaluate accuracy and computational performance are ad hoc and in...
متن کاملBenchmarking 301: Advanced practice metrics provide actionable insight.
College students are familiar with class rating systems. Freshmen take courses designated in the 100 range, sophomores take 200 courses and upperclassmen, who are more advanced in their studies, take courses in the 300 or 400 range. In the context of benchmarking, basic metrics provide the key measures you need to manage a practice. Using metrics that are standardized on a per full-time-equival...
متن کاملService for Calculation of Performance Metrics of Indoor Localization Benchmarking Experiments
This report presents the service for calculation of performance metrics of indoor localization benchmarking experiments. By performance metrics we mean point and room level accuracy of location estimation, response time and power consumption of the system under test. The service is developed in order to standardize the way performance metrics of indoor localization benchmarking experiments are ...
متن کامل